A framework for the theoretical evaluation of XML retrieval

This article presents a theoretical framework to evaluate XML retrieval. XML retrieval deals with retrieving those document components, the XML elements, that specifically answer a query. In this paper, theoretical evaluation is concerned with the formal representation of qualitative properties of retrieval models. It complements experimental methods by showing the properties of the underlying reasoning assumptions that decide when a document is about a query. We define a theoretical methodology based on the idea of ‘aboutness’ and apply it to current XML retrieval models. This allows comparing and analyzing the reasoning behaviour of XML retrieval models experimented within the INEX evaluation campaigns. For each model we derive functional and qualitative properties that qualify its formal behaviour. We then use these properties to explain experimental results obtained with some of the XML retrieval models.